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Abstract 


It has been acknowledged that evidence-driven practices may lead schools to 
improved instructional practices, student learning, or organizational improve- 
ment; still the evidence is underused by the teachers or school leaders. This study 
focuses on analyzing how to strengthen the evidence-driven school improvement 
in school-university partnership programs. Five schools learnt over a period of one 
school year in collaboration with the university coaches how to collect evidence 
in classroom and organizational level for improvement process. The results of 
our study illustrate profiles of the schools based on the usage of data-informed 
evidence, research-based evidence, or both to make decisions in the instructional 
and organizational level. Enablers and barriers of data use from the perspective of 
organizational, user, and data characteristics to implement evidence-driven prac- 
tices are discussed. 


Keywords: data-informed evidence, research-based evidence, evidence-driven 
school improvement, school-university partnership 


1. Introduction 


The Estonian Lifelong Learning Strategy 2020 [1] aims to implement a learning 
and teaching approach that supports each learner’s individual and social develop- 
ment, learning skills, creativity, and entrepreneurship in the work of all levels 
and types of education. To achieve this demanding goal, new teaching practices 
should be developed in collaboration between universities and schools. This also 
means that every school should focus on their students’ individual needs—instead 
of implementing already existing approaches, new solutions should be created 
or modified to fit into local context. When adapting new teaching and learning 
methods, important questions arise: what is the impact of these approaches and 
what other factors are influencing the outcomes. 

In this new situation, schools continuously develop their practices, analyze the 
needs of teachers’ and find ways for their professional development. Hansen and 
Wasson [2] have pointed out that there is a need to change teachers’ professional 
development format—instead of traditional participation in training courses, 
teachers should be supported in developing and improving their existing practice 
through teacher inquiry. Nowadays, capacity building, inquiry-oriented prac- 
tice, and data-driven decisions are considered as central themes of educational 
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improvement [3, 4]. Concepts like practitioner research and teacher inquiry have 
been widely used for several decades—yet schools still face difficulties in using 
evidence for school improvement processes [1]. 

In the age of big data, it is difficult to imagine any educational improvement 
that does not include data as a key pillar [6]. Developing evidence-driven school 
improvement processes through school-university collaboration is one option for 
helping schools work with evidence. Therefore, school-university joint programs 
are initiated and the Future School Program was launched in Estonia. The aim of 
the Future School Program is to support whole-school innovation and sustainable 
improvement of teaching practices by enhancing the teaching and learning culture 
through school-university co-creation of new methodologies and implementation 
of evidence-driven innovation. 

In this chapter, we analyze how to strengthen the evidence-driven school 
improvement in school-university partnership program. Following questions are 
discussed: 


e How evidence-driven school improvement is actualized in school development 
programs? 


e What are the enablers and barriers of using evidences in school development 
program? 


2. Evidence-driven school improvement: theoretical underpinnings 


Nowadays, educational innovation is not only the “business” of scholars— 
practitioners are actively involved and discussions about educational improve- 
ments revolve around the importance of evidence and data. Different authors use 
distinct terminology [7] evidence-informed education [8], evidence-informed 
practices [9], evidence-based practice [10], evidence-based education [11, 12], 
data-based decision-making [13, 14], data-informed practice [15], data-driven 
decision-making [16, 17], data-based decision-making [18], data use [7, 19-21] 
and practice-informed evidence [22]. The main idea behind these concepts seems 
to be concurrent; however, the use of different terms is not incidental. One of 
the broadest explanation has been given by Davies [23], who sees evidence-based 
education as a set of principles and practices, which can alter the way people 
think about education, the way they go about educational policy and practice, 
and the basis upon which they make professional judgments and deploy their 
expertise—but it is not the provider of readymade solutions to the demands 
of modern education. In the following sections, we compare and analyze how 
different concepts supplement each other and how the evidence-based improve- 
ment can be identified for the schools. 

To start with, we need to unravel the concepts of evidence as they are widely 
used. Evidence is a kind of information, which points to the truth or validity of a 
claim and is the joint starting point for all authors; opinions differ on how truth 
or validity is achieved. It is assumed that the main source of evidence practitio- 
ners should consider when making decisions in social science research, namely 
experimental research and randomized controlled trials [10, 11, 24]. The idea 
that research can make a major contribution to improving practice stems from 
the assumption that it is systematic and rigorous and provides explicit evidence, 
which can be assessed objectively [10]. It can be concluded that one sub-concept 
of evidence-based education concentrates on implementation of research results, 
especially implementation of these teaching techniques and methods, which have 
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been found to have a positive effect on students’ assessment results. In the follow- 
ing, we distinguish this sub-concept as a research-based school development. 

Research-based evidence as a source for school development and teachers’ 
professional development has been criticized from different aspects. The disap- 
proval of research-based evidence has been argued with the nature of research, its 
generalizability, and objectivity. It is recognized that professional judgments cannot 
be made without taking into consideration the value-based foundation of educa- 
tion [11]. However, research findings merely inform practitioners about what the 
general outcomes are of different kinds of decisions [24], and there are a variety 
of formal and informal sources of information that also contribute to the decision- 
making process [10, 24]. Schools and teachers cannot wait until the valid and 
reliable research results say how to implement new teaching practices. 

Evidence-based education operates at two levels. First is to utilize existing 
evidence from worldwide research and literature on education and associated 
subjects [23]. This gives a broader base for professional knowledge-in action [15]. 
The second level is to establish sound evidence where existing evidence is lacking 
or of questionable, uncertain, or weak in nature [23]. It requires acquiring, using, 
critiquing, and creating the evidence base by the lived experience of observing 
and assessing students in particular contexts on a regular basis [15]. This type of 
professional knowledge relies on multiple values, tacit judgment, local knowledge, 
and skill; research usually cannot supply what the notion of evidence-based practice 
demands of it—specific and highly reliable answers to questions about what works 
and what does not [10]. In this case, the basis for innovating instruction is the data 
what the context offers. The data about the students, their background, their previ- 
ous achievements, as well as teaching processes, and school organizational existence 
is wide and the potential of this data is unused. 

The definition of data is broad. The focus is on raw data that must be organized, 
filtered, and analyzed to become information, then combined with stakeholder 
understanding and expertise to become actionable knowledge. The data not only 
enclose student test results, but also any other form of structurally collected 
qualitative or quantitative data on the functioning of the school, such as outcomes, 
inputs, processes, and perceptions [13, 25]. In short, data are the information that is 
collected and represent some aspect of schools [26]. If the evidence incorporates the 
question and the answer, the data comprehend only the question and the potential 
of the answer. The evidence incorporates the interpretive and evaluative elements, 
which are missing from the data. In conclusion, we distinguish the second sub- 
concept of evidence-driven school improvement as a data informed. 

In addition to the data-informed and research-based dimensions of evidence, 
the distinction of the outcomes can be identified [7]. The expected outcomes of the 
evidence usage describe the goals for which the evidence is used, more specifically, 
the aspect of the school culture which is expected to be improved and changed 
according to the conclusions made from the evidence. 

Discussions of evidence-based or evidence-informed practice refer to teachers, 
their classroom activities, and interactions with students [7, 9]. The data and evi- 
dence use are implemented with the goal of improving instruction. The quality of 
teachers’ instruction is an important influence on student achievement, and using 
data for improving instruction can enhance student achievement [13]. 

Data and evidence can also be used to inform decision-making in school man- 
agement and leadership levels. This process is often called data-based or data-driven 
decision-making [13, 16]. Data-driven decision-making is the purposeful process of 
selecting, gathering, and analyzing relevant data to define school problems, develop 
alternatives, estimate outcomes of the alternatives, and choose the preferred alter- 
native [16]. Data do not objectively guide decisions on their own—but people do. To 
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do so, they select particular pieces of data to negotiate arguments about the nature 
of problems as well as the potential solutions [14]. The use of data is not only a mat- 
ter of new competencies and skills, it is more about the new culture to arise. Good 
things do not happen thanks to data—it should be supported by data-informed 
leaders. Leaders should take the responsibility to evaluate what types of data are 
useful and for what purposes [17]. Organizational practices have an important role 
in affecting the way that people in organizations think and work, so it is possible to 
shift patterns of practice by creating organizational supports and incentives that 
give greater prominence to the consideration of research findings and their implica- 
tions [8]. In sucha case, the data and evidence can be used for school development 
purposes and it refers to schools using data to improve themselves; for instance, 
student satisfaction surveys and exam results can be used to evaluate the extent to 
which the school is achieving its goals [13]. The processes of decision-making and 
interpretation happen in parallel; this way, there is potentially a higher coherence 
among the data, the decision, contextual factors, as well as the risk of misinterpre- 
tation or biased interpretation. 

In conclusion, we have identified two dimensions of the concept of evidence- 
driven school improvement (Figure 1). One of the dimensions is the input dimen- 
sion, which refers to different inputs of the evidence: the evidence can be data 
informed or research based. The data-informed evidence can appear from assess- 
ment results, characteristics of teaching staff, national or school surveys, etc. The 
research-based evidence can be the result of some experimental study or qualitative 
study on teachers’ behavioral patterns. The second dimension characterizes the 
output of evidence: whether the evidence is influencing decisions made for school 
development, incorporating the organizational aspects like the structures, com- 
munication, or decisions made for the improvement of instruction by the teacher, 
usually in the interaction with the student and used educational method. 

Studies of data use have analyzed the factors influencing evidence-driven school 
improvement, and based on the synthesis of recent studies [13], it can be concluded 
that these factors are organizational characteristics, user characteristics, and data 


Evidence for school development 


National Research on 
surveys, public patterns of 
databases, interactions, 
characteristics of structures, meta- 
teaching staff analysis 
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Figure 1. 
Dimensions of evidence-driven school improvement and some examples. 
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characteristics. Organizational factors include the shared vision and clear norms 
for data usage, encouragement by the school leader, possible expert support, time, 
and conditions provided for collaboration between teachers. Data use depends on 
the user characteristics of teachers. In order to use data, teachers need to have the 
knowledge and skills needed to analyze and interpret different forms of data; they 
need to understand the quality criteria for data use and data-use concepts; and they 
need skills to diagnose student-learning needs and adjust instruction accordingly. 
Data characteristics are identified as access to student relevant data, and the usabil- 
ity and high quality of data. It is important to note that these factors can be enablers 
or barriers depending on the goal of the data use. The study [13] shows that data use 
for school development is influenced by organizational and data characteristics, but 
data use for improving teaching and learning is influenced by organizational and 
user characteristics. 

Additionally, the evidence-driven school improvement implemented in school- 
university partnerships is influenced by the character of the relationship. The 
partnership can be as two types of relationships between schools and universities: 
one type of partnership can be labeled as transactional and refers to a relationship, 
which is driven by individual purposes—in this case, the organizations remain 
unchanged; the second type is transformational partnership, where the parties 
come together to pursue a common purpose and create the possibility of growth and 
change through mutual interaction as they apply their resources to address complex 
problems [27]. Studies [7, 17, 25, 28] investigating strategies of school-university 
partnerships for supporting evidence-driven school improvement have identified 
four key domains: (1) human support, (2) leadership, (3) technology support, and 
(4) designed routes. 

One possibility to offer human support is to use coaches. In order for coach- 
ing on data use to be effective, teachers needed to believe that the coach possesses 
strong interpersonal skills, content, and pedagogical knowledge that would be 
useful for them to learn. Facilitation of coaches includes assessing teachers’ needs, 
modeling how to interpret and act upon data, and observing teachers while they 
attempt to engage in the data-use process. Another possibility is to support profes- 
sional development, but from previous studies, it is evident that the structured 
training in how to use data is not common in schools. A third approach to human 
support is networking with a university: either the researcher guides the process of 
data analysis and brings a theoretical framework to the practice or relies primarily 
on workshops and ongoing consultancy. 

Schools make efforts to have technology support: data systems that organize and 
analyze interim assessment data, and data warehouses with current and historical 
student data. It is acknowledged that the trainings for school teams on data use are 
rare and focus primarily on technological support and how to access the data man- 
agement system. Technological support needs to be combined with other strategies. 

School leadership—principals are key players in facilitating data use among 
teachers, they play an important role in allocating resources and time to enable 
teachers to use data effectively. Their espoused beliefs about data use are critical as 
well, so they help set the tone for data use among in school teams. For the school 
leader, it is important to have a whole school perspective on the improvement 
initiated. If the instructional and organizational improvements are not aligned, it is 
confusing and unmotivating for the teachers to participate. It is important to com- 
municate for the teachers why the evidence is being collected in classroom level and 
how it helps to monitor the big picture of the improvement and data are not used 
to blame-and-shame teachers. The evidence-driven school improvement cannot be 
implemented without data-literate and research-wise school leader, so the crucial 
target to support strategies is the leaders in schools. 
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Schools are required to follow norms and designed routes—specific data-driven 
decision-making practices—when developing their school improvement plans or 
for teachers to follow when using data to guide instruction. One of the primary 
ways that is used to build teachers’ capacity to use data is providing structured time 
for collaboration. This includes adoption of data-discussion protocols in order to 
ensure that discussions about data occurred and that actions were taken on the basis 
of these conversations. 

Factors that influence the successful implementation of a school development 
program with the aim to support evidence-driven school improvement have been 
studied. Schools are more successful, if the entire school team participates in the 
program, the school staff is stable and the school leader provides their teachers with 
sufficient time and materials. It is concluded that school leaders and trainers should 
pay attention to developing clear guidelines and agreements on the execution of 
evidence-driven school improvement activities [18]. 


3. Methodology 
3.1 Context: overview of the school improvement program 


The research context is formed around the school improvement program estab- 
lished at the Tallinn University. The program aims to support the evidence-driven 
improvement in Estonian schools for improving teaching and learning culture. Five 
schools applied (Table 1), based on their interest, to join the program in 2018/2019. 
Each school team consisted of 5-6 members, whereas 1-2 of them where members 
of the management and each school formulated their own student-centered goal for 
the improvement they aimed to achieve. 


3.1.1 Evidence-driven improvement process 


In the first phase of the program, each school prepared an action plan for 
improvement. Before creating the action plan, an analysis of the state of the school, 
built on existing evidence, had to be carried out. Some of the schools used data col- 
lected at the national level (satisfaction surveys, students’ study results, and exist- 
ing research studies) to understand the current situation, defining the problem, 


School type Improvement goal 


Teachers: 111 Public kindergarten-basic Teachers’ collaboration implementing 
Students: 811 school collaboration-days 


S1 

S2 Teachers: 68 Public secondary school development of students’ learning to learn 
Students: 769 (up to 12 grade) skills 

s3 Teachers: 19 Public basic school (up to9 | students’ motivation to learn through 
Students: 169 grade) systematic integration of the lessons and 

outside of the classroom activities 

S4 Teachers: 60 Public secondary school students’ engagement in extracurricular 
Students: 519 (up to 12 grade) activities 

S5 Teachers: 19 Public basic school (up to 9 | integrating real-life situations to classroom 
Students: 155 grade) activities in 7th grade 


Table 1. 
Characteristics of schools participating in the school improvement program. 
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and formulating the action plan. When analyzing the evidence, three school teams 
changed their initial goals because they did not find clear evidence about the 
problem they thought the school had or they identified another problem based 

on the evidence. During program activities, schools had to monitor and reflect on 
their own activities to understand their improvement processes. Each team agreed 
upon their own approach and tools for monitoring and data collection, which were 
discussed with their university coach. In addition to the regular monitoring, each 
school had to design their own action research plan, carry out the study in a class- 
room setting, analyze the collected data, and come up with suggestions on how the 
data will be used in the next decision-making steps. 


3.1.2 School-university partnership 


The program consisted of elements of human support, support for leadership, 
and designed joint activities. The school team—where the school leader was a 
compulsory member—participated in monthly seminars, where the next steps of 
the program were explained through theoretical underpinnings and practical sug- 
gestions. The seminars were used in the program, because the studies have shown 
that supporting professional development is essential in raising data-literacy skills 
of educational practitioners [25]. The networking aspect of the seminars is also 
effective to support for schools. Between seminars, the school team was supported 
by their university assigned coach. The coach is recognized as one of the key ele- 
ments in offering human support [25]. Each step was scaffolded with the special 
task designed according to principles of change management and evidence-driven 
improvement. Data use can be improved by data-use routines, ensuring that it is a 


recurrent and patterned interaction that guides how people engage with each other 
and data [7]. 


3.2 Data collection and analysis 


We followed the case-study approach, which has been acknowledged as an 
appropriate method in educational studies about evidence use [28]. Case studies 
do not aim to produce generalizable theories, but aim to provide practical wisdom, 
which is “about understanding and behavior in specific situations” [29]. That was 
also the aim of our study—to better understand the collaborative practices support- 
ing schools in implementing evidence-driven school improvement. 

Data were collected throughout the program and after the completion of the 
program. A variety of data gathering techniques that are summarized in Table 2 
were used. 

Data were analyzed based on the framework from theoretical underpinnings, 
where different dimensions of evidence use for school improvement were defined 
(Table 3). Instructional-level decision-making refers to the teachers’ decisions 
to improve their own teaching, assessment, feedback, etc. Organizational-level 
decisions refer to the decisions made by school management or school improvement 
team to improve school-level processes, practices, curriculum design, etc. 

Evidence-driven practices of the five cases were classified according to nine 
possible profiles of evidence-driven school improvement. These profiles were cre- 
ated according to criteria defined from the dimensions of evidence-driven school 
improvement. The criteria were the following: 


e Whether the school collected (a) data-informed evidence, (b) research- 
based evidence, or (c) both. We classified the school as using data-informed 
evidence when the data were collected by the school or made available for 


Pedagogy in Basic and Higher Education - Current Developments and Challenges 


Semi-structured, face-to-face focus group interviews with 
the teachers and individual interviews with the principals to 
understand the co-creation practices and challenges faced. 


Interviews with 
the teachers and 
principals 


5 school principals 
14 teachers from 5 schools 


5 teams from 5 schools, 
altogether 36 posts 


Monthly Google form questionnaire consisting of 8 questions, but 4 
reflections by the relevant for this study to identify the co-creation practices. 
school teams 


5 school reports, 
presentation 


Documents Materials created by the schools as part of their inquiry 
created by the activities (action research results and data collection 
school instruments) 


Table 2. 
Overview of data collection. 


Instructional and 
organisational level 
decision making 


Instructional level 
decision making 


Organisational level 
decision making 


Data-informed decision Data-informed decision 
making in instructional and | making in organisational 


Data-informed 
decision making in 


Data-informed 
evidence 


Data informed 
and research- 
based evidences 


instructional level 


Data-informed and 
research-based 
decision making in 
instructional level 


organisational level 


Data-informed and 
research-based decision 

making in instructional and 
organisational level 


level 


Data-informed and 
research-based decision 
making in organisational 
level 


Research-based decision Research-based decision 
making in instructional and | making in organisational 
organisational level level 


Research-based 
decision making in 
instructional level 


Research-based 
evidence 


Table 3. 


Profiles of the cases based on dimensions of evidence-driven school improvement. 


the schools by other stakeholders, and analysis was done by the school team 
based on their own research and improvement interest. We classified the 
school as using research-based evidence when the data have been collected, 
analyzed, and published by researchers, and the results are used by schools in 
their improvement process. 


e Whether the school analyzed the results with the goal (a) to improve school 
management, (b) to improve instruction in the classroom, or (c) both. The 
school was classified to use evidence on management level when the school 
team made decisions about communication, professional development, work 
organization, procedures, etc. We classified the evidence as used for the 
instructional improvement if the conclusions and recommendations were 
targeted toward teachers and their activities. 


4. Results 


Evidence-driven practices as part of the school improvement were tightly 
embedded into the different phases of program activities. Next, the schools’ 
practices to actualize the evidence-driven school improvement, the challenges, and 
enablers of the process will be analyzed and discussed. The aim was to understand 
the following: to what extent schools used evidence collected from wider research, 
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whether they collected or analyzed data based on their own research interests, and 
was the results used in organizational-level or instructional(teacher-student) -level 
decision-making processes. 

Based on teams’ reflections, interviews and analysis of the documents schools 
were profiled as follows (Table 4): usage of data-informed evidence, research-based 
evidence, or both to make decisions in the instructional level or organizational level 
or both. 


4.1 Data-informed decision-making in organizational level 


The aim for school 1 was to improve the teachers’ collaboration and through that 
improve the students’ learning experience, for that a new initiative was established 
as “collaboration day.” Based on the reflections and document analysis, the school 
team focused mainly on collecting data from teachers and students to understand 
the usability and effectiveness of the collaboration format—dquestionnaire for the 
teachers and students after each collaboration day, students’ self-analysis, and 
observation sheets. Evidence regarding well-established methods and theoretical 
underpinnings were less emphasized by this school in their improvement process. 
The main outcome for the school team was that the intervention supported teach- 
ers’ collaboration and integration of subjects: 


Teachers are more involved in collaborative learning: the number of teachers 
participating in more than 1-2 integration projects has increased by about 20%; 
teachers make more suggestions to colleagues for collaboration. 


The majority of the decisions based on the collected and analyzed data were 
done in management level: improving the format of the collaboration days, identi- 
fying the needs for teacher training, sharing practices, and supporting documenta- 
tion of the integration projects. 


4.2 Research-based and data-informed decision-making in organizational level 

School 4 focused on students’ engagement in extracurricular activities. 
Interventions were carried out in teacher-student level and students’ engagement 
was analyzed with observation sheets. Students’ motivation was analyzed and 


teachers’ feedback was collected with self-analysis: 


We analyzed what emerged from the teachers’ work analysis and students' motiva- 
tion questionnaire. 


Theories and studies regarding students’ learning motivation to support engage- 
ment were used as evidence to plan the interventions and data collection: 


We used motivation theories, introduced by the university, to plan our intervention. 


Decisions were made mainly in the management level: observation process and 
techniques need to be improved: 


Not everything is always visible—how to go on with the improvement of the 
observation sheet. 


More focus on supporting teachers’ sharing of experiences and good practices 
was put. 
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Cases Instructional level Instructional and Organisational level 
decision making organisational level decision | decision making 
making 
Data-informed School 3 ae 1 
evidence 


Research-based 
evidence 


Data-informed and School 2 School 4 
research-based School 5 
evidence 


Table 4. 
The schools evidence-driven profiles. 


4.3 Research-based and data-informed decision-making in instructional 
and organizational level 


The aim of the school 2 was to implement different learning strategies to support 
the development of students’ learning to learn skills. For monitoring the process, 
several data collection techniques were used: teachers’ empowerment survey, survey 
about teachers’ understanding of learning to learn skills, and teachers’ interviews 
about different strategies. Students’ self-analysis about the learning process was 
carried out; students learning skills and reading strategies were tested. Evidence 
from national-level satisfaction surveys was used when planning the activities and 
later analyzed: 


National survey 2018 was used to plan the activities; National survey 2019 was 
used to analyze the state of the school. 


Approved training programs about reading and meaningful learning were used 
when designing interventions in collaboration with the university team. To support 
the collaborative culture, a teachers’ professional learning community was initiated 
and research on teachers’ professional community was used to support teachers’ 
collaborative learning. Teachers in this group were also studied: 


We conducted interviews with the teachers’ part of the learning community. 


Decisions were made in management level (training and management support 
for teachers’ to implement the new strategies to support students’ learning to learn 
skills) and in instructional processes (new strategies will be implemented and 
students’ self-analysis process more systematically enhanced). 

School 3 aimed to raise the students’ motivation to learn through more sys- 
tematic integration of the lessons and outside of the classroom activities. Self- 
determination theory was used as a research ground in different activities: 


In designing and conducting action research, we relied on self-determination theory. 


To analyze the effectiveness of the interventions, data were collected with the 
students’ survey after each intervention (based on self-determination theory) and 
teachers’ feedback. Evidence from the national-level students’ satisfaction survey 
was used for planning interventions. Decisions regarding the future activities were 
made in students’ level: focusing on explaining the goals of different learning activi- 
ties to enhance the meaningfulness, enhancing students’ skills to give feedback: 
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Students may not have taken the feedback seriously; the purpose of the survey 
should be better explained to the students. 


In management-level lesson, observations based on self-determination theory 
were developed. 

The aim of the school 5 was to implement the meaningful learning experience 
for the seventh grade students through integrating more real-life situations to class- 
room activities. For data collection, an instrument was created to analyze to what 
extent students understand what they learn and how it supports their professional 
growth. Also, all the students were tested with scientific tests: 


Grade 7 students took a motivation test and a social skill and learning to-learn skill 
test. 


Students and teachers gave weekly feedback, and teachers analyzed the students’ 
evaluation sheets: 


In addition to the paper-based feedback, we also received feedback from students 
electronically, which makes feedback for teachers more concise. 


Also oral feedback was collected from teachers and students for more in-depth 
analysis of the new experiences. Some evidence about the studies on integration of 
subjects was also used. To some extent, research results were also read by the team: 


We read some research about integration of the subjects. 


Decisions were made mainly on management level: improving evaluation sheets, 
reformulating learning outcomes to make them easier for the students to under- 
stand. In the instructional level, teachers will focus more in the future to create 
shared understanding with the students about what learning outcomes mean and 
what students are actually expected to learn: 


The teacher does not refer to the relation of the subject's learning outcome to every- 
day life, the result—teacher formulates the links between the learning outcomes 
together with the students. 


Also the plan to create individual learning paths for the students is in the focus 
for the future activities. 

Our analysis indicates that all five schools participating in our program focused 
on collecting data and finding research evidence on the management level and three 
schools worked with evidence in the instructional processes. Four schools out of 
five focused on improving students’ learning experience; one school focused on 
teachers’ collaboration, but still with the aim to implement integration projects to 
improve teaching practices in the classroom level. It can be also concluded that all 
schools used data as part of their own studies to understand the effectiveness of 
the interventions, but the usage of the research evidence did not happen systemati- 
cally in all of the cases. Schools collected data from both students and teachers; 
the instruments were mainly prepared by the schools themselves. In a few cases, 
additional data were collected with research instruments proposed by the univer- 
sity (testing the skills of the students for instance). Decisions made based on the 
data and research results were mainly focused on management level: improving 
everyday processes, data collection techniques, formalizing methodologies, and 
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better supporting teachers’ collaboration. Some important decisions were also made 
on the student level: enhancing feedback skills, goal-setting of learning activities, 
enrichment of classroom activities, etc. 


4.4 The enablers and barriers using evidence in school improvement program 


Schildkamp and colleagues [5, 13] have proposed several factors influencing data 
use by school teams; they distinguish data use for accountability, school development, 
and instruction. In our research, we mainly focused on data use for school improve- 
ment and classroom-level instruction. Deriving from Schildkamp et al. [5, 13], we 
analyze the enablers and barriers of data use from the perspective of organizational, 
user and data characteristics. 

Organizational characteristics include the shared vision, which includes a joint 
understanding about the nature of good teaching, student learning, and ways to 
evaluate the student learning. As our program focused on school improvement, 
building shared understanding about the change and ways to monitor the process 
were crucial. Schildkamp et al. [13] emphasize that effective data use also requires 
collaboration—teachers should share and discuss their students’ results and their 
own functioning with students, parents, and teachers. In our case, all of the schools 
focused on improving teachers’ collaboration and different solutions were found 
to find time to share experiences as part of the program activities. However, school 
2—which created a teachers’ professional learning community where the collected 
data were analyzed and results discussed—stood out among others for its evidence- 
driven school improvement practices. In our study, it was learnt that for the schools, 
it was difficult to design and conduct empirical studies (in action-research form) 
on their own (It is a very complex process for the school to develop research-based 
inquiry.) This was emphasized by the school that collaborated more tightly with the 
university experts to carry out research activities. On the other hand, same schools 
used more systematically research-based evidence in their improvement process 
than schools who used less university support in their activities. Therefore, the 
collaboration between the school team and university became very important in our 
study. Research data were used, but schools needed help in this regard, because it 
was challenging for the schools to understand what research data they could use and 
for what purposes and how to adapt the research-based solutions for their school 
settings. In our program, it was the role of the coaches to found experts, refer to 
the relevant studies, share validated tests and observation sheets to adapt, collect 
research data, etc. This relates well with Schildkamp et al. [13] user characteristics 
as well—data literacy of the teachers is something that needs promotion. It is not 
easy for the teachers to have the inquiry mindset, skills to collect data, interpret, 
and act based on the data. Mandinach [21] has concluded that pedagogical data 
literacy is the ability to transform information into actionable instructional 
knowledge and practices by collecting, analyzing, and interpreting all types of 
data to help specify educational steps by combining an understanding of data with 
standards, disciplinary knowledge and practices, curricular knowledge, pedagogi- 
cal content knowledge, and an understanding of how children learn. Once teachers 
are prepared to work with the data, data characteristics—quick and convenient 
access to accurate data—also become very important. In our study, data-collection 
instruments were mainly prepared in collaboration with the university coaches and 
experts or by school teams themselves. It can be concluded that planning the data 
collection in collaboration with the university is something that schools can apply 
during the program activities. However, analyzing data quickly for feedforward 
purposes is something that needs further planning. For instance, school 4 who 
developed paper-based observation sheet learned that such documentation format 
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does not support instant decision-making for classroom-level instruction. And 
school 1 changed their paper-based surveys to electronic surveys in the middle of 
the program for more efficient data analysis purposes. 


5. Further perspectives and practical implications 


Our study indicated that in school-university partnerships, schools are able to 
acquire easier the mindset of evidence-driven improvement based on data collec- 
tion, analysis by school team, or evidence from theoretical and methodological 
underpinnings. However, there are some aspects that need to be considered. 


5.1 Human support 


A coach has been suggested as one possibility to offer human support, which 
was also applied in the current program, and it can happen in school-university 
partnerships where the university coach guides the process. Facilitation by 
coaches includes assessing teachers’ needs, modeling how to interpret and act 
upon data and observing teachers while they attempt to engage in the data-use 
process. It is recommended to design trainings for the school team with the 
following learning outcomes: learning the capabilities of the data system, under- 
standing and using a cycle of instructional improvement, avoiding common data 
analysis mistakes, data transparency and safety, fostering a culture of data use, 
interpreting data in context, and using data to modify instruction. From the per- 
spective of human support in the school-university partnership, our experience 
highlights the importance of the university coach. The school teams recognized 
the coaches help with practical questions and choices. This opens the discussion 
on the role of the coach in the school-university partnership. The university coach 
is often conceptualized in the literature as a data coach [7] or researcher [30] who 
pays attention primarily on evidence use. It may be too narrow of an approach if 
the final aim is to find and co-create innovative teaching and leading practices for 
school improvement. Yet, in our case, the profiles of the schools evidence-driven 
school improvement show that finding and selecting appropriate research-based 
evidence needs strengthening in the school improvement program. Also the 
main focus of the coaches was on bringing in theoretical frameworks, fostering a 
culture of evidence use and understanding the cycle of inquiry. The data analysis 
mistakes or accuracy was less emphasized by the schools. However, it was men- 
tioned by one of the schools that they actually would like to get feedback if their 
inquiry design, data collection, and analysis are adequate. 


5.2 Technology support 


When technology training exists, it often focuses primarily on technological 
support and how to access the data management system. Studies show that schools 
pay efforts to have data systems that organize and analyze interim assessment data 
and data warehouses with current and historical student data. Our study indicated 
that elements to scaffold teachers to conduct teacher-led inquiry in the technology- 
enriched classroom as suggested by Hansen and Wasson [2] can be better sup- 
ported. In our program, the data were collected rather traditionally—tests, surveys 
and questionnaires, mainly, and paper-based observation sheets. Focusing more 
on process-oriented data collection—with a variety of tools and efficient ways for 
classroom observations—timely access to students’ learning results might influ- 
ence the use of data for improving the classroom instruction. The growing use of 
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technology as part of teachers’ practice opens up the possibility for a change from 
researcher-centered studies to teacher-centered approaches to inquiry [2]. 


5.3 Leadership 


School principals are key players in facilitating data use among teachers—they 
play an important role in allocating resources and time to enable teachers to use evi- 
dence effectively. Their espoused beliefs about data use are critical, as they help to 
set the tone for data use in school teams. School leaders also have access to a variety 
of data, performance indicators, and study results; making these available for the 
teachers is important to enhance the data culture in the organization. However, we 
recognized that during the program, schools mostly used the data they gathered by 
themselves and the use of data gathered by or for the national or municipality level 
was used rarely. This raises the question of the capabilities to interpret such data 
by the school team, and capabilities to support and coach this interpretation by the 
university coaches. Moreover, our coaches could recognize some hesitations and 
doubts for using such data by the school teams because of the meaningfulness of the 
data gathered in this manner. We recognize the effective use of national data as an 
improvement area for the school development program. 


5.4 Norms and designed routes 


The schools are required to follow specific data-driven decision-making prac- 
tices when developing their school improvement plans or for teachers to follow 
when using data to guide instruction. Providing structured time for collaboration is 
one of the primary ways that schools try to build teachers’ capacity to use data. This 
includes adoption of data-discussion protocols in order to ensure that discussions 
about data occurred and that actions were taken on the basis of these conversations. 
Our program focused on understanding how can we better support schools in work- 
ing with the data; in the next iteration of the program, we can more systematically 
focus on supporting the development of practices to create norms and routes for 
more systematic evidence-driven school improvement. 

Our study demonstrated that in school-university partnership, when schools 
are scaffolded, evidence-driven practices are more widely adopted by the schools 
as part of the school improvement process. However, we also learned that the need 
for teachers to obtain complex data skills is becoming more and more important. 
Understanding about the inquiry process is just one angle of the challenge; also the 
understanding of how to read, interpret, critically evaluate, and act based on data 
is as important. In this iteration, the program did not systematically emphasize 
designing practices for collecting evidence from data and from the research, which 
could be better supported in the future. Also, we learned that schools understand 
quite well how to improve the practices in the school level based on collected 
evidence. Synergy between instructional-level data collection and decision-making, 
and organizational-level improvement can, however, be enhanced. In the future, it 
is important to analyze the impact of using classroom data in novel pedagogical and 
assessment approaches, and for teacher’s professional development to determine if 
it changes the students’ learning. 

Our study also informs us how to improve initial teacher education and school 
principals’ preparation in Estonia. The main practical implication is rooted in 
the dimensions of evidence-driven school improvement. Currently, in initial 
teacher education, students are expected to carry out action research project 
during their internship period. Individually they learn how to collect data in the 
teaching process. They do not experience how their collected data from classroom 
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interventions could feed the school improvement process and what is the relation 
between classroom-level evidence with school-level evidence. It can be concluded 
that it needs strengthening the dimension of evidence for school improvement in 
the initial teacher training. Additionally, current initial teacher training tends to 
prepare future teachers to collect action research data rather traditionally through 
surveys and interviews, but the usage of the learning analytics solutions as part of 
the inquiry could enable to monitor the practices more efficiently. Simultaneously, 
in principals’ training program, topics like evidence-driven school improvement 
and schools’ self-evaluation are rather theoretical. However, school principals need 
skills how to collect, analyze, interpret, and integrate data about instructional inter- 
ventions conducted by teachers to plan improvements in school-level processes. 
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